11. Exercise: Preventing Resource Leaks

External Sort: Part 2

Remember that sharding program you wrote in the last exercise? It's time to finish the job!

This time, you will write another Java program that reads in the sorted shard files and combines them into a single large, sorted text file.

For this part of the exercise you will be opening all the shard files at once (but not reading their entire contents all at once — remember that there are too many words to fit into memory!). Because you will be using a bunch of open BufferedReaders at the same time (one for each shard file), you should focus on using try-finally and/or try-with-resources to make sure all the files are correctly closed so that your program will not have any resource leaks.

Inside the try part of a try-finally block, create the List of BufferedReaders: one for each input Path. Without modifying the shard files, merge them together into a single text file whose location is specified by the outputPath.

To do this, you should store words in a PriorityQueue<WordEntry>, but make sure the priority queue never contains more entries than there are input files (the WordEntry helper class has already been implemented for you). Remember, the whole point of doing a distributed merge sort is that there are too many words to fit into memory!

Once you have all the BufferedReaders constructed, in the same try block, use a try-with-resources to build a BufferedWriter to write the output file. This code should consist of a loop that continually calls PriorityQueue#poll() to get the next alphabetical word to write, and then adds more words into the PriorityQueue. Each time you write a word to the output file, the PriorityQueue should be "replenished" with a new word from the same BufferedReader that produced the word you just wrote. The loop should continue until the PriorityQueue is empty.

In the same loop, write the words to the output file using a BufferedWriter. Make sure to use try-with-resources to close the writer when you are done writing.

Finally, in the finally part of the try-finally block, be sure to close all the BufferedReaders.

Running the Solution

Finally, run the solution to make sure it works:

javac MergeShards.java
java MergeShards shards/ sorted.txt

You should see a single text file, sorted.txt with 10,000 words in it, sorted in alphabetical order!